{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "92010cd5",
      "metadata": {},
      "source": [
        "# 2.3 Optimizing prompts to improve the quality of responses from Q&A bots\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b0b18130",
      "metadata": {},
      "source": [
        "## 🚄 Preface  \n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "4a33d28f-9014-414a-a7b5-b016e12d4bb0",
      "metadata": {},
      "source": [
        "The Q&A bot from the previous section can already answer questions about company rules and regulations, but this is only the first step in building an excellent user experience. Users often expect more personalized and precise interactions. This section will introduce techniques in prompt engineering, teaching you how to improve or control the output of the Q&A bot by optimizing prompts—for example, adjusting tone, standardizing formats, or even enabling it to handle tasks like text summarization, inference, and transformation."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "37a39a58",
      "metadata": {},
      "source": [
        "## 🍁 Goals\n",
        "\n",
        "After completing this section , you will know about:\n",
        "\n",
        "* How prompt frameworks and templates work\n",
        "* Prompt techniques and their best practices\n",
        "* How to apply LLMs in engineering to handle various tasks"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5b0e80fa",
      "metadata": {},
      "source": [
        "## 1. Previous content recap\n",
        "\n",
        "In the previous section, through the RAG method, the LLM has already obtained the company's private knowledge. For ease of invocation, this was encapsulated into several functions and saved in chatbot/rag.py. You can now quickly invoke it using the following code:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "id": "7aa6756d-47a4-4b11-8074-d992d6ce327b",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2025-01-15T06:49:03.577636Z",
          "start_time": "2025-01-15T06:49:03.573248Z"
        }
      },
      "outputs": [],
      "source": [
        "from chatbot import rag\n",
        "import os\n",
        "from config.load_key import load_key"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "id": "f23f2c04-c229-4f5f-b2cd-50c8a45672e4",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2025-01-15T06:49:09.717475Z",
          "start_time": "2025-01-15T06:49:09.713681Z"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Your configured API Key is: sk-4b*****\n"
          ]
        }
      ],
      "source": [
        "os.environ[\"BASE_URL\"] = \"https://dashscope.aliyuncs.com/compatible-mode/v1\"\n",
        "\n",
        "# Load API key\n",
        "load_key()\n",
        "print(f'Your configured API Key is: {os.environ[\"DASHSCOPE_API_KEY\"][:5]+\"*\"*5}')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "id": "91138451-fbc1-49ff-a268-2ece341b10bd",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2025-01-15T06:49:11.770743Z",
          "start_time": "2025-01-15T06:49:11.611835Z"
        }
      },
      "outputs": [],
      "source": [
        "# Load index\n",
        "# The previous section has already built the index, so the index can be loaded directly here. If you need to rebuild the index, you can add a line of code: rag.indexing()\n",
        "index = rag.load_index(persist_path=\"knowledge_base/test\")\n",
        "query_engine = rag.create_query_engine(index=index)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "id": "1d2d8d11-5a07-441c-9cd6-6ba10bf80603",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2025-01-15T06:49:16.961802Z",
          "start_time": "2025-01-15T06:49:16.959373Z"
        }
      },
      "outputs": [],
      "source": [
        "# Define the question-answering function\n",
        "def ask_llm(question, query_engine):\n",
        "  streaming_response = query_engine.query(question)\n",
        "  streaming_response.print_response_stream()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b1715f36",
      "metadata": {},
      "source": [
        "## 2. Prompt optimization to improve response quality \n",
        "\n",
        "In the previous section, you obtained the management tools used internally by the company through the RAG method. However, one of your colleagues hopes that the Q&A bot not only provides the tool name, but also includes a link to the tool. You could ask all your colleagues to include a note like \"provide the download address\" in their questions, but this approach is inefficient. You decide to enhance the program by automatically supplementing user questions with additional requirements for the response content."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "id": "b12a310a-660c-4cd3-9d23-8819bdefb8bb",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2025-01-15T06:49:34.992776Z",
          "start_time": "2025-01-15T06:49:29.417135Z"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "For project management, the company can use Asana for task tracking and Jira for technical team coordination. These tools will help streamline workflows and enhance collaboration across teams.\n",
            "\n",
            "- **Asana**: Ideal for coordinating tasks and projects among instructional designers and other teams. You can sign up or learn more at [Asana’s official website](https://asana.com/).\n",
            "\n",
            "- **Jira**: Perfect for technical teams to manage daily standups and project sprints. More information and downloads are available at [Jira’s official website](https://www.atlassian.com/software/jira)."
          ]
        }
      ],
      "source": [
        "question = \"What tools should our company use for project management?\"\n",
        "instruction = \"If it's a tool consultation question, be sure to provide the download link.\"\n",
        "new_question = question + instruction\n",
        "ask_llm(new_question, query_engine)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "806fadd6-673a-4542-b3e2-fe36310b30ad",
      "metadata": {},
      "source": [
        "After adjusting the way prompts were provided, the Q&A bot's responses became more aligned with your colleagues' requirements. Therefore, the choice of prompt significantly determines the quality of the LLM's responses. Next, you can refer to some prompt frameworks to construct your prompts."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c006a5ea-5693-4805-9d2c-ccc0d2214939",
      "metadata": {},
      "source": [
        "## 3. Prompt framework\n",
        "\n",
        "### 3.1 Basic elements\n",
        "\n",
        "When communicating with an LLM, you can think of it as a person who has undergone \"socialization training.\" The way you interact should mirror how humans exchange information. Your requirements must be clear and unambiguous. The clearer and more precise your questiom (or prompt), the better the large language model will understand the core of the query, and the more likely its response will meet your expectations. Generally speaking, the following elements should be clarified in a prompt, and form a prompt framework that can help you construct a complete and effective prompt:\n",
        "\n",
        "|Element|Meaning|\n",
        "|----|----|\n",
        "|Objective|Clearly specify what task the LLM is required to complete, allowing it to focus on specific goals.|\n",
        "|Context|Background information about the task, such as operational processes or task scenarios, clarifying the scope of the discussion for the LLM.|\n",
        "|Role|The role the LLM plays, or the voice or writing style that the model should use, clarifying the expected emotional tone of the response.|\n",
        "|Audience|Clarify the specific audience the LLM is targeting, constraining its response style.|\n",
        "|Sample|Provide specific examples for the LLM to refer to; it will abstract solutions and focus on specific formats from these samples.|\n",
        "|Output Format|Specify the format of the output, output type, and range of enumerated values. You can also state content and information that should not be included in the output, which can be further clarified with examples.|\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "dc35341a-3d7c-4881-81e3-935d42a1dcb2",
      "metadata": {},
      "source": [
        "Of course, in addition to the prompt framework discussed above, many problem analysis paradigms can be used to help you articulate clear and specific requirements. For example, SWOT analysis and 5W2H analysis offer structured approaches to thinking through your needs and expressing them more effectively. Additionally, you can also write a meta-prompt—a prompt designed to optimize or enhance another prompt—to guide the LLM in refining your initial input.\n",
        "\n",
        "By using such a meta-prompt, you can ask the model to:\n",
        "* Clarify ambiguous statements\n",
        "* Expand vague instructions into detailed guidelines\n",
        "* Add context, tone, format, and examples where necessary\n",
        "* Ensure the final prompt aligns with best practices for interacting with AI models\n",
        "\n",
        "This self-reflection approach improves the quality of your prompts, while deepening your understanding of how to communicate effectively with AI systems."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "3c618996",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "#### 1. Optimized Prompt:\n",
            "\"Craft a compelling promotional copy (approximately 150-200 words) for our company’s new book titled [Book Title]. The book focuses on [brief description of the book's content, e.g., innovative strategies for educational content development, practical tips for teacher training, or data-driven approaches to improving learning outcomes]. The promotional copy should highlight the key benefits and unique selling points of the book, such as its practical insights, real-world applications, and expert authorship. Emphasize how the book can help educators, trainers, and learners achieve better results. Use a persuasive and engaging tone that resonates with educational professionals and enthusiasts. Ensure the copy includes a call-to-action encouraging readers to purchase the book or learn more about it.\"\n",
            "\n",
            "#### 2. Example Input:\n",
            "\"The book is titled 'Mastering Educational Content Development.' It provides comprehensive guidance on creating high-quality educational materials, including strategies for integrating technology, designing effective teacher training programs, and leveraging data analysis to improve learning outcomes. The book is written by experienced educators and content developers.\"\n",
            "\n",
            "#### 3. Example Output:\n",
            "\"Discover the ultimate guide to creating impactful educational content with 'Mastering Educational Content Development.' This comprehensive resource offers invaluable insights and practical strategies for educators and content creators. Learn how to integrate cutting-edge technology into your curriculum, design effective teacher training programs, and leverage data analysis to enhance learning outcomes. Written by seasoned experts in the field, this book is your go-to resource for transforming educational materials into powerful tools for success. Whether you're a classroom teacher, a content developer, or an educational administrator, 'Mastering Educational Content Development' will equip you with the knowledge and skills to make a lasting difference. Get your copy today and start revolutionizing your approach to education!\"\n",
            "\n",
            "#### 4. Optimization Notes:\n",
            "- Added specific details about the book's content and title.\n",
            "- Defined the word count and target audience.\n",
            "- Highlighted key benefits and unique selling points.\n",
            "- Specified the tone and style to be persuasive and engaging.\n",
            "- Included a clear call-to-action to encourage readers to take the next step."
          ]
        }
      ],
      "source": [
        "prompt_expand = \"\"\"\n",
        "    You are a Prompt Optimization and Expansion Assistant. Your primary role is to receive an arbitrary user input prompt — which may be vague, incomplete, or lacking in detail — and **refine and expand** it into a more structured, precise, and actionable instruction that can be clearly understood and executed by large language models.\n",
        "\n",
        "    Given the input prompt, you should generate an optimized version according to the following instructions:\n",
        "\n",
        "    Your task is to enhance the clarity of the original prompt while preserving its core intent. The resulting optimized prompt should:\n",
        "    - Be semantically richer and more descriptive.\n",
        "    - Include relevant context, constraints, or formatting instructions as needed.\n",
        "    - Specify desired tone, style, or audience (if applicable).\n",
        "    - Suggest example inputs and outputs where appropriate to guide model behavior.\n",
        "\n",
        "    ---\n",
        "\n",
        "    ### Output Format:\n",
        "\n",
        "    Please respond with the following structure:\n",
        "\n",
        "    #### 1. Optimized Prompt:\n",
        "    [Here goes the enhanced and expanded version of the original prompt.]\n",
        "\n",
        "    #### 2. Example Input (if applicable):\n",
        "    [Provide one or more example(s) of what a user might input based on the prompt.]\n",
        "\n",
        "    #### 3. Example Output (based on the example input):\n",
        "    [Show what kind of output would be expected from the model when using this prompt with the given example input.]\n",
        "\n",
        "    #### 4. Optimization Notes:\n",
        "    [Briefly explain how you improved the original prompt — e.g., added clarity, specified format, included context, etc.]\n",
        "\n",
        "    ---\n",
        "\n",
        "    ### Example:\n",
        "\n",
        "    #### Original Prompt:\n",
        "    \"Write a story about animals.\"\n",
        "\n",
        "    #### Optimized Prompt:\n",
        "    \"Compose a short narrative (approximately 300–500 words) about a group of forest animals who must work together to solve a problem. The story should include at least three distinct animal characters, a central conflict or challenge, and a resolution that highlights themes such as cooperation, courage, or environmental awareness. Use descriptive language to set the scene and convey character emotions. Aim for a tone suitable for children aged 6–10.\"\n",
        "\n",
        "    #### Example Input:\n",
        "    None required — the model generates content autonomously.\n",
        "\n",
        "    #### Example Output:\n",
        "    Once upon a time in Greenleaf Forest, a wise old owl named Oliver noticed that the river had stopped flowing. Concerned, he called a meeting with his neighbors: Mina the mouse, Benny the bear, and Tilly the turtle. Together, they discovered a fallen tree blocking the stream. With teamwork and determination, they cleared the path and restored the water supply. From then on, the animals made sure to keep their home safe and healthy...\n",
        "\n",
        "    #### Optimization Notes:\n",
        "    - Added specific narrative elements (characters, setting, conflict, theme).  \n",
        "    - Defined word count and target audience.  \n",
        "    - Clarified tone and style expectations.  \n",
        "    - Introduced emotional and moral dimensions to enrich storytelling.\n",
        "\n",
        "    ---\n",
        "\n",
        "    Please apply this process to any user-submitted prompt and return your response using the above template.\n",
        "\n",
        "    The original prompt is:\n",
        "\n",
        "\"\"\"\n",
        "\n",
        "def input_enhanced_llm_output(questions):\n",
        "    expanded_input = prompt_expand + questions\n",
        "    return ask_llm(expanded_input, query_engine)\n",
        "\n",
        "question = \"Please help me write a promotional copy for our company's new book. The general content of the book is xxx(will be given later)\"\n",
        "input_enhanced_llm_output(question)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5e1fa6d9",
      "metadata": {},
      "source": [
        "### 3.2 Prompt template\n",
        "\n",
        "When developing large language model applications, it's not ideal to let users write prompts based on a framework. Instead, you can use elements from various prompt frameworks to build a structured prompt template. A prompt template allows you to predefine key information, such as the role of the LLM and any important guidelines, which helps guide the model’s behavior. Developers only need to adjust input parameters within the template to create consistent and standardized applications for LLMs."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a84e29a3-a017-47c4-907e-2ff897333d1d",
      "metadata": {},
      "source": [
        "In the RAG chatbot created using LlamaIndex, there is a default prompt template that looks like this:\n",
        "\n",
        "- The default template can be viewed using code. You can refer to [the code on the LlamaIndex official website.](https://docs.llamaindex.ai/en/stable/examples/prompts/prompts_rag/) The original LlamaIndex prompt template is:  \n",
        "\n",
        "\n",
        "\n",
        "```text\n",
        "Context information is below.\n",
        "---------------------\n",
        "{context_str}\n",
        "---------------------\n",
        "Given the context information and not prior knowledge, answer the query.\n",
        "Query: {query_str}\n",
        "Answer:\n",
        "```\n",
        "\n",
        "Both `context_str` and `query_str`  are placeholders that get replaced during the retrieval and querying process. The `context_str` is filled with relevant information retrieved from the vector database, while `query_str` is replaced with the user's actual question.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "78ff8788-4d67-47b8-a4a2-3f3a85e6b0c5",
      "metadata": {},
      "source": [
        "Since the default template is general-purpose , it may not be suitable for all scenarios. For example, you might adjust the tone, add specific instructions, or define the expected format of the response. This customization ensures that the model’s output aligns more closely with your requirements.\n",
        " \n",
        "You can use the following sample code, where prompt_template_string represents the new prompt template:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "id": "05aea996-fdc8-4a35-922f-288313881fa3",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2025-01-15T07:52:22.442975Z",
          "start_time": "2025-01-15T07:52:22.435217Z"
        }
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<llama_index.core.query_engine.retriever_query_engine.RetrieverQueryEngine at 0x3415c6e10>"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Build prompt template\n",
        "prompt_template_string = (\n",
        "    \"You are the company's customer service assistant, you need to answer users' questions concisely.\"\n",
        "    \"\\n【Notes】：\\n\"\n",
        "    \"1. Answer user questions based on contextual information.\\n\"\n",
        "    \"2. Only answer the user's question, do not output other information\\n\"\n",
        "    \"The following is reference information.\"\n",
        "    \"---------------------\\n\"\n",
        "    \"{context_str}\\n\"\n",
        "    \"---------------------\\n\"\n",
        "    \"Question: {query_str}\\n.\"\n",
        "    \"Answer: \"\n",
        ")\n",
        "\n",
        "# Update prompt template\n",
        "rag.update_prompt_template(query_engine, prompt_template_string)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fa99f324",
      "metadata": {},
      "source": [
        "## 4. Techniques for writing effective prompts\n",
        "\n",
        "In Section 3.1, some key elements of prompt design were introduced. This section will explain specific prompt techniques in detail, focusing on practical scenarios and how to apply the elements of a prompt effectively.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f1b3eb55-646d-41dd-b801-152d76205bbe",
      "metadata": {},
      "source": [
        "### 4.1 Clearly express requirements and use delimiters\n",
        "\n",
        "Clearly stating your requirements ensures that the content generated by the LLM is highly relevant to the task at hand. These requirements typically include the **task objective**, **context**, and **background information**.\n",
        "\n",
        "Using **delimiters** helps LLMs focus on specific goals, avoid ambiguous interpretations, and reduce unnecessary processing. Delimiters also help structure your prompt in a way that improves readability and model understanding.\n",
        "\n",
        "Here are somecommonly used delimiters in English prompt engineering:\n",
        "\n",
        "| Delimiter Type      | Example                         | Use Case / Description |\n",
        "|---------------------|----------------------------------|--------------------------|\n",
        "| Triple Quotes         | `\"\"\"...\"\"\"`                      | For multi-line strings or input blocks |\n",
        "| Triple Hashes         | `### Section ###`                | To mark headings or instruction sections |\n",
        "| Double Angle Brackets | `<<example>>...<</example>>`     | To define custom tags for variables or examples |\n",
        "| XML-like Tags         | `<input>...</input>`             | For structured formatting and content separation |\n",
        "| Dashes                | `--- Section ---`                | To separate paragraphs or logical blocks |\n",
        "| Equal Signs           | `=== Output Format ===`          | Similar to headings; often used for formatting clarity |\n",
        "\n",
        "> ⚠️ Note: Avoid using symbols that appear frequently in your content as delimiters to prevent confusion. For example, if square brackets `[]` are commonly used  in your input text, they may not be suitable as delimiters.\n",
        "\n",
        "---\n",
        "\n",
        "### ✅ Example Usage in an English Prompt\n",
        "\n",
        "```text\n",
        "### Instruction ###\n",
        "You are a helpful assistant. Please summarize the following article.\n",
        "\n",
        "=== Article Content ===\n",
        "\"\"\" \n",
        "Artificial intelligence (AI) is a branch of computer science that aims to create systems capable of performing tasks that typically require human intelligence.\n",
        "\"\"\"\n",
        "\n",
        "### Output Format ###\n",
        "Please provide a summary in no more than 50 words.\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "id": "8918731d-49b6-4132-9ba5-2dce3eee360c",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2025-01-15T07:52:32.192934Z",
          "start_time": "2025-01-15T07:52:24.619420Z"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Welcome to the New Employee Training Camp!\n",
            "\n",
            "Embark on an exciting journey designed to not only equip you with the essential skills and knowledge but also to foster a sense of community and collaboration. This training camp is your gateway to becoming a valued member of our dynamic team. Here, you will:\n",
            "\n",
            "- **Dive Deep into Our Culture:** Understand the values, mission, and vision that drive us forward.\n",
            "- **Build Strong Foundations:** Gain hands-on experience and in-depth knowledge of our products and services.\n",
            "- **Forge Lasting Connections:** Engage in team-building activities that promote camaraderie and teamwork.\n",
            "- **Learn from the Best:** Receive mentorship from experienced professionals who are eager to share their insights and expertise.\n",
            "- **Navigate Your Path:** Participate in orientation sessions that guide you through your role and responsibilities, setting you up for success.\n",
            "\n",
            "Join us in this enriching experience and take the first step towards a rewarding career with us!"
          ]
        }
      ],
      "source": [
        "question = \"\"\"\n",
        "### Instruction ###\n",
        "Expand and polish the text enclosed in XML tags below to make the copy vivid, creative, and appealing to new employees. \n",
        "The tone should be professional yet welcoming, suitable for onboarding materials.\n",
        "\n",
        "=== Original Text ===\n",
        "<text>\n",
        "New Employee Training Camp Activity\n",
        "</text>\n",
        "\n",
        "=== Output Requirements ===\n",
        "- Enhance the content with engaging language.\n",
        "- Add context about the purpose and benefits of the training camp.\n",
        "- Include key elements such as team-building, mentorship, and orientation.\n",
        "- Keep the final output concise and motivational.\n",
        "\"\"\"\n",
        "\n",
        "ask_llm(question, query_engine)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "62ae4c28-a2e6-4ad3-995b-7af2d78b19d1",
      "metadata": {},
      "source": [
        "From the above execution results, it is evident that the prompt effectively communicates the task requirements: to expand and enhance the provided text.\n",
        "\n",
        "The core theme of the task is \"**New Employee Training Camp Activities,**\" and the intended audience is \"**new company employees.**\" In the prompt, the delimiter <text>...</text> is used to clearly encapsulate and isolate the original content for processing.\n",
        "\n",
        "By structuring the prompt with clear instructions, appropriate context, and standardized delimiters, the model is better guided to generate output that not only preserves the original meaning but also becomes more vivid, engaging, and tailored to the target audience."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e7e4407d-3cff-4f00-a39b-b8ddfd7cce1d",
      "metadata": {},
      "source": [
        "### 4.2 Define Role and Audience\n",
        "\n",
        "The **role** refers to the identity the LLM should assume in a specific scenario, such as an expert, advisor, or assistant. The **audience** is the target user group for which the model's output is intended, such as general consumers or students. Defining these elements helps developers tailor the tone, depth, and structure of the content accordingly.\n",
        "The example below demonstrates how different system roles affect the style and content of the generated output. \n",
        "\n",
        "> Pass the prompt template into `prompt_template_string`, and the user question into `question`, then call the `ask_llm` function to execute it."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "79e5f128-8b02-40ca-b789-d577b6e38a3a",
      "metadata": {},
      "source": [
        "<table>\n",
        "  <thead>\n",
        "    <tr>\n",
        "      <th width = '80px'>Role</th>\n",
        "      <th width = '500px'>LLM Algorithm Engineer</th>\n",
        "      <th width = '500px'>Elementary School Teacher</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody valign = 'top'>\n",
        "    <tr>\n",
        "      <td>Prompt Template</td>\n",
        "      <td>\"You are a senior large language model algorithm engineer. Please answer the user's question based on the provided information\\n\"\n",
        "\"---------------------\\n\"\n",
        "\"{context_str}\\n\"\n",
        "\"---------------------\\n\"\n",
        "\"Question: {query_str}\\n.\"\n",
        "\"Answer: \"</td>\n",
        "      <td>\"You are an elementary school teacher. Please answer the user's question based on the provided information\\n\"\n",
        "\"---------------------\\n\"\n",
        "\"{context_str}\\n\"\n",
        "\"---------------------\\n\"\n",
        "\"Question: {query_str}\\n.\"\n",
        "\"Answer: \"</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <td>User Question</td>\n",
        "      <td>Qwen-VL</td>\n",
        "      <td>Qwen-VL</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <td>Output Answer</td>\n",
        "      <td>Qwen-VL is a multimodal pre-trained model introduced by Alibaba Cloud. It combines text and image processing capabilities, enabling it to understand and generate text descriptions related to images. Simply put, Qwen-VL can be regarded as an artificial intelligence assistant that can \"describe images in words.\" It not only understands the content of images but also generates corresponding textual descriptions or answers questions related to the images. This technology has broad application prospects in various fields, such as automatic image tagging, visual question-answering systems, and content-based image retrieval. For new media operations, tools like Qwen-VL can help improve content creation efficiency, enhance user experience, and provide users with richer and more diverse ways of presenting information.</td>\n",
        "      <td>Kids, today we’re going to meet a new friend named Qwen-VL!\\nQwen-VL is like a very smart robot that has \"eyes\" 👀 and a \"brain\" 🧠. It can \"see\" pictures and videos and understand their meaning, just like us!\\nFor example, if you show it a picture of a kitten playing with a ball of yarn, it can not only recognize that it’s a kitten 🐱 but also tell you that it’s playing with a ball of yarn 🧶. Even better, you can ask it, \"What color is the kitten?\", and it will tell you the color of the kitten.\\nIt can also understand the content in videos. For instance, if you show it a video of kids playing soccer ⚽️, it can tell you who is kicking the ball, where the ball goes, and even guess what might happen next!\\nQwen-VL is like a child who is still learning. It keeps getting smarter and improving, and in the future, it will become even more intelligent, capable of doing more things and helping us better understand the world!</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>  \n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "10f06bea-a34b-4e7a-b2b5-918af4c1ebdb",
      "metadata": {},
      "source": [
        "Sometimes, when developers design applications using LLMs, they need the model to produce structured data as output so it can be used by downstream systems to complete the full application workflow. While LLMs typically generate free-form natural language, they are fully capable of producing structured outputs such as JSON, XML, or CSV formats—as long as the desired format and requirements are clearly specified in the prompt.\n",
        "In the following example, based on the prompt template from Section 3.2, we will act as a Q&A bot to assist in reviewing the quality of company-developed documentation, and return the results in **JSON format**:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "id": "e6692496-b6e4-4d75-aaa2-ef38b0c24f0c",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2025-01-15T06:49:49.007218Z",
          "start_time": "2025-01-15T06:49:46.080617Z"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{\n",
            "  \"label\": 0,\n",
            "  \"reason\": \"There is a typo in 'langauge', which should be 'language'.\",\n",
            "  \"correct\": \"Delimiters are special symbols that help large language models (LLMs) identify which parts of the prompt should be considered a complete unit of meaning.\"\n",
            "}"
          ]
        }
      ],
      "source": [
        "question_task = \"\"\"\n",
        "[Task Requirements]\n",
        "You will be given a sentence or paragraph. Your task is to review it for any spelling or grammatical errors. \n",
        "If there are errors, you should identify them, explain the issue, and provide a corrected version. \n",
        "\n",
        "---\n",
        "[Output Requirements]\n",
        "Please output only in JSON format. Do not include markdown or code blocks.\n",
        "The JSON must contain:\n",
        "- \"label\": 0 if an error is found, 1 if no error.\n",
        "- \"reason\": A brief explanation of the error (if any).\n",
        "- \"correct\": The corrected version of the text (or the original if no error).\n",
        "\n",
        "---\n",
        "[User Input]\n",
        "The following is the user input. Please review it:\n",
        "\"\"\"\n",
        "\n",
        "question_doc = \"Delimiters are special symbols that help large langauge models (LLMs) identify which parts of the prompt should be considered a complete unit of meaning.\"\n",
        "\n",
        "question = question_task + question_doc\n",
        "\n",
        "ask_llm(question, query_engine)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ebe59cd3-a241-4b28-902e-7ca35966e6a7",
      "metadata": {},
      "source": [
        "As shown in the results of the above example, the prompt `question_task` specified that the output format should be JSON format, and clearly defined the content requirements. The model  generated responses in the required format. This consistent and structured output makes it easier to integrate the model into existing systems.\n",
        "\n",
        "On news websites, blog platforms, or internal knowledge-sharing platforms within enterprises, user-edited or published articles may contain typos, grammatical errors, or even sensitive information. However, traditional manual review methods are prone to oversight. In this case, LLMs can be integrated to automatically review content. If an article is flagged for serious grammatical errors or high-risk sensitive words, its priority for revision will be set to \"high.\" For articles with minor issues, the priority for revision can be set to \"low.\" This approach reduces labor costs and enhances the efficiency and accuracy of the system.\n",
        "\n",
        "Of course, there are many other applications similar to this scenario. Developers can analyze system bottlenecks or focus on data-intensive tasks to explore more use cases for LLMs."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "75f32b7a-9b47-40f1-82cb-e889a3b48048",
      "metadata": {},
      "source": [
        "### 4.4 Providing few-shot examples\n",
        "\n",
        "In the example from Section 4.3, the prompt specified the output format, and the LLM  generated formatted content. However, if we want the model's output  to not only be correctly formatted but also maintain consistency in style and structure, providing a few examples as references can be highly effective. This is like giving the model a \"reference book.\" \n",
        "\n",
        "In the following code example, let’s first observe the LLM's output without any examples:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "id": "afcca16d368baf38",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2025-01-15T07:20:42.883097Z",
          "start_time": "2025-01-15T07:20:18.423060Z"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "# Handmade Keychain Making Tutorial\n",
            "\n",
            "## Introduction\n",
            "Making handmade keychains is a fun and creative activity that allows you to personalize gifts or accessories. This tutorial will guide you through the steps to create a simple yet stylish keychain using basic materials and tools.\n",
            "\n",
            "## Materials Needed\n",
            "- **Beads**: Assorted sizes and colors\n",
            "- **Jump rings**: Small metal rings for attaching components\n",
            "- **Key ring**: To attach the keychain to keys\n",
            "- **Wire**: Thin, flexible wire for stringing beads\n",
            "- **Pliers**: Round-nose and flat-nose pliers for manipulating wire and jump rings\n",
            "- **Scissors**: For cutting wire\n",
            "- **Clasp** (optional): For adding a secure closure\n",
            "\n",
            "## Step-by-Step Instructions\n",
            "\n",
            "### Step 1: Gather Your Materials\n",
            "Before you start, lay out all your materials on a clean, flat surface. Ensure you have everything listed above within reach.\n",
            "\n",
            "### Step 2: Cut the Wire\n",
            "Using your scissors, cut a piece of wire that is about 6 inches long. This length should be sufficient for most keychains, but you can adjust it based on the number of beads you plan to use.\n",
            "\n",
            "### Step 3: String the Beads\n",
            "Thread your chosen beads onto the wire. You can mix and match colors and sizes to create a unique design. If you want a symmetrical look, plan your bead pattern before starting.\n",
            "\n",
            "### Step 4: Secure the Beads\n",
            "Once you have strung all the beads, use your pliers to create a loop at one end of the wire. This will prevent the beads from falling off. To do this:\n",
            "1. Bend the wire at a 90-degree angle about 1/4 inch from the end.\n",
            "2. Wrap the wire around the base of the bend several times to form a secure loop.\n",
            "3. Trim any excess wire with your scissors.\n",
            "\n",
            "### Step 5: Attach the Jump Ring\n",
            "Open a jump ring using your pliers. Slide the looped end of your beaded wire through the jump ring, then close the jump ring securely. This will connect your beaded section to the key ring.\n",
            "\n",
            "### Step 6: Add the Key Ring\n",
            "Attach the key ring to the jump ring. Make sure it is securely closed to prevent the keychain from coming apart.\n",
            "\n",
            "### Step 7: Optional Clasp\n",
            "If you want to add a clasp for extra security, open another jump ring and attach it to the opposite end of your beaded wire. Then, attach the clasp to this jump ring and close it securely.\n",
            "\n",
            "### Step 8: Final Touches\n",
            "Inspect your keychain to ensure all connections are secure. Adjust any beads or loops if necessary. Once you are satisfied, your handmade keychain is ready to use!\n",
            "\n",
            "## Tips and Variations\n",
            "- **Personalize It**: Add charms or pendants that reflect the recipient's interests.\n",
            "- **Experiment with Colors**: Try different color combinations to create a variety of styles.\n",
            "- **Use Different Materials**: Consider using leather, fabric, or metal components for a different look.\n",
            "\n",
            "## Conclusion\n",
            "Creating a handmade keychain is a rewarding project that can be tailored to suit any style. Follow these steps to make a beautiful and functional keychain that you can be proud of. Happy crafting!"
          ]
        }
      ],
      "source": [
        "question_task = \"\"\"\n",
        "### 📝 Task Requirements\n",
        "You are to generate content based on the user's topic, following the **style and structure** of the example provided below. Do not add any extra text or explanations — return only the final Markdown-formatted output.\n",
        "\n",
        "---\n",
        "### 🧩 Output Format Requirements\n",
        "Please ensure your response strictly follows this format:\n",
        "- Use Markdown syntax (headings, lists, bold, links, image placeholders, etc.)\n",
        "- Keep the tone informative and easy to follow\n",
        "- Structure content in a step-by-step or tutorial format\n",
        "- Avoid markdown code blocks around the final output itself\n",
        "\n",
        "---\n",
        "### 👤 User Input Topic\n",
        "The following is the user's requested topic:\n",
        "\"\"\" \n",
        "\n",
        "question_doc = \"Handmade Keychain Making Tutorial\"\n",
        "\n",
        "question = question_task + question_doc\n",
        "\n",
        "ask_llm(question, query_engine)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "30f61d316e005c4f",
      "metadata": {},
      "source": [
        "The model output a tutorial for making handmade keychains, but the content was not concise enough. If you want the model to output content in a specific style and structure—such as focusing only on the theme, list of materials, and step-by-step instructions—you can provide a few examples to guide it. This helps the model \"imitate\" the desired format and produce more consistent and structured results."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "id": "c1e0f67f703e7ba",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2025-01-15T06:50:39.297041Z",
          "start_time": "2025-01-15T06:50:24.071562Z"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "### Making Handmade Greeting Cards\n",
            "\n",
            "# Handmade Greeting Card Tutorial\n",
            "\n",
            "## Materials Needed\n",
            "- Colored cardstock\n",
            "- Scissors\n",
            "- Glue stick\n",
            "- Markers or colored pencils\n",
            "- Decorative elements (stickers, glitter, ribbons)\n",
            "\n",
            "## Steps\n",
            "1. **Choose Your Cardstock**\n",
            "   - Select a piece of colored cardstock that matches the occasion (e.g., birthday, holiday).\n",
            "\n",
            "2. **Cut the Cardstock**\n",
            "   - Fold the cardstock in half to create a card shape. Ensure the edges are aligned neatly.\n",
            "\n",
            "3. **Design the Front**\n",
            "   - Use markers or colored pencils to draw a design on the front of the card. You can draw flowers, animals, or any other pattern you like.\n",
            "\n",
            "4. **Add Decorations**\n",
            "   - Enhance your design by adding stickers, glitter, or ribbons. Place them carefully to complement your drawing.\n",
            "\n",
            "5. **Write a Message**\n",
            "   - Open the card and write a personal message inside. Use a pen or marker to make it stand out.\n",
            "\n",
            "6. **Final Touches**\n",
            "   - Review your card to ensure everything is secure and looks as intended. Make any last-minute adjustments if needed.\n",
            "\n",
            "7. **Done!**\n",
            "   - Your handmade greeting card is ready to be given to someone special. Enjoy the satisfaction of creating something unique and heartfelt."
          ]
        }
      ],
      "source": [
        "question_task = \"\"\"\n",
        "### 📝 Task Requirements\n",
        "You are to generate content based on the user's topic, following the **style and structure** of the example provided below. Do not add any extra text or explanations — return only the final Markdown-formatted output.\n",
        "\n",
        "---\n",
        "### 🧩 Output Format Requirements\n",
        "Please ensure your response strictly follows this format:\n",
        "- Use Markdown syntax (headings, lists, bold, links, image placeholders, etc.)\n",
        "- Keep the tone informative and easy to follow\n",
        "- Structure content in a step-by-step or tutorial format\n",
        "- Avoid markdown code blocks around the final output itself\n",
        "\n",
        "---\n",
        "### 📚 Example Content\n",
        "<example>\n",
        "### Example 1: Making a Simple Bookmark\n",
        "# Simple Bookmark Making Tutorial\n",
        "\n",
        "## Steps\n",
        "1. Choose a piece of colored cardstock.\n",
        "2. Use a pencil to draw a rectangle on the cardstock, with dimensions approximately 2 inches x 6 inches.\n",
        "3. Cut out the rectangle along the pencil lines.\n",
        "4. Personalize the bookmark by decorating it with stickers.\n",
        "5. Done! You now have a unique bookmark.\n",
        "</example>\n",
        "\n",
        "---\n",
        "### 👤 User Input Topic\n",
        "The following is the topic requested by the user:\n",
        "\"\"\"\n",
        "\n",
        "question_doc = \"Making Handmade Greeting Cards\"\n",
        "\n",
        "question = question_task + question_doc\n",
        "\n",
        "ask_llm(question, query_engine)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e95447ee9772539d",
      "metadata": {},
      "source": [
        "From the above example results, it's clear that the LLM has fully adhered the provided example and generated content with a consistent structure and style. When specifying the output format in a prompt, it is recommended to provide a few examples for the LLM to reference. This practice helps ensure that the model's output is more stable, consistent, and aligned with your expectations."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "53569b47-1619-4527-a9c8-b53f5e5bdf8d",
      "metadata": {},
      "source": [
        "### 4.5 Giving the model \"thinking\" time\n",
        "\n",
        "For some complex tasks, even well-structured prompts may  not be sufficient ffor the LLM to complete the task effectively. In such cases, you can guide the model by allowing it to \"think\" step by step, encouraging it to provide intermediate reasoning before arriving at a final answer.. This approach, known as the chain-of-thought (CoT) method, improves performance on tasks that require logical or mathematical reasoning.\n",
        "\n",
        "For example, consider a scenario where you ask the LLM to solve a math problem. The correct answer is $10,500. Initially, you might use a simple prompt:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "id": "3569f469-c601-406a-a26d-fb0a74157c68",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2025-01-15T06:50:42.416991Z",
          "start_time": "2025-01-15T06:50:41.378053Z"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "9000 yuan"
          ]
        }
      ],
      "source": [
        "question = \"\"\"\n",
        "[Background Information]\n",
        "An educational training institution (hereinafter referred to as the \"company\") incurred the following main expenses in the 2023 fiscal year:\n",
        "To provide classes for students in different cities, the company's teachers traveled on business trips 5 times during the year, each trip lasting one week. The specific expenses are as follows:\n",
        "   - Transportation and accommodation fees: average 1600 yuan/trip\n",
        "   - Teaching materials procurement costs: At the beginning of the year, the company purchased a batch of teaching materials for a total price of 10,000 yuan, which is expected to last for 4 years.\n",
        "\n",
        "[Problem Description]\n",
        "Based on the above background information, complete the following tasks:\n",
        "Calculate the total travel expenses for the year due to teacher business trips, including the amortized cost of teaching materials.\n",
        "\n",
        "[Output Requirements]\n",
        "Provide only the total travel expenses directly, without any other information. \"\"\"\n",
        "ask_llm(question, query_engine)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "edf9078a-f716-4537-b003-a90d70f988b5",
      "metadata": {},
      "source": [
        "As shown, the initial response is incorrect. To improve accuracy, you can use the CoT method by prompting the LLM to explain its reasoning step by step.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "id": "92cb1715-305c-43ee-bd5e-274d2556ad56",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2025-01-15T06:50:55.556383Z",
          "start_time": "2025-01-15T06:50:43.720893Z"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "To calculate the total travel expenses for the year, including the amortized cost of teaching materials, follow these steps:\n",
            "\n",
            "1. **Calculate the total transportation and accommodation fees:**\n",
            "   - Number of trips: 5\n",
            "   - Average cost per trip: 1600 yuan\n",
            "   - Total transportation and accommodation fees = 5 trips * 1600 yuan/trip = 8000 yuan\n",
            "\n",
            "2. **Calculate the annual amortized cost of teaching materials:**\n",
            "   - Total cost of teaching materials: 10,000 yuan\n",
            "   - Expected lifespan: 4 years\n",
            "   - Annual amortized cost = 10,000 yuan / 4 years = 2500 yuan\n",
            "\n",
            "3. **Calculate the total travel expenses for the year:**\n",
            "   - Total transportation and accommodation fees: 8000 yuan\n",
            "   - Annual amortized cost of teaching materials: 2500 yuan\n",
            "   - Total travel expenses = 8000 yuan + 2500 yuan = 10500 yuan\n",
            "\n",
            "Therefore, the total travel expenses for the year, including the amortized cost of teaching materials, is 10,500 yuan."
          ]
        }
      ],
      "source": [
        "question = \"\"\"An educational training institution (hereinafter referred to as the \"company\") incurred the following major expenses in the 2023 fiscal year:\n",
        "To provide classes for students in different cities, the company's teachers traveled on business trips 5 times throughout the year, with each trip lasting one week. The specific expenses are as follows:\n",
        "   - Transportation and accommodation fees: an average of 1600 yuan per trip\n",
        "   - Teaching materials procurement costs: At the beginning of the year, the company purchased a batch of teaching materials for a total price of 10,000 yuan, which is expected to last for 4 years.\n",
        "   \n",
        "### Problem Description\n",
        "Based on the above background information, complete the following tasks:\n",
        "Calculate the total travel expenses for the year caused by teacher business trips, including the amortized cost of teaching materials.\n",
        "\n",
        "### Output Requirements\n",
        "Please derive step by step to calculate the total travel expenses.\"\"\"\n",
        "\n",
        "ask_llm(question, query_engine)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b0ec9cdf-d6e1-469f-9e4c-f91fdf7f1c3e",
      "metadata": {},
      "source": [
        "After optimizing the prompt, the LLM was able to generate an accurate and detailed response. This demonstrates how adding a CoT to prompts can significantly improve the quality of reasoning-based outputs.\n",
        "\n",
        "While CoT is a powerful technique, there are other methods to enhance the model’s thinking process, such as tree of thoughts (ToT) andgraph of thoughts (GOT). However, current LLMs still rely heavily on CoT for structured reasoning. As AI technology advances, models are gradually moving toward more sophisticated multi-agent systems (MAS), which can handle complex tasks more efficiently. You can explore this further in <2_6_Extend_FAQ_Bot_Capabilities_with_Plugins>."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6214d05f-8154-44f7-943b-7f04196a3275",
      "metadata": {},
      "source": [
        "## 5. Using LLMs for intent recognition\n",
        "\n",
        "After learning about prompt engineering techniques, your colleagues have realized that the Q&A bot can not only be used to look up information, but also assist  in reviewing documents and translating content. While the Q&A bot is generally effective at identifying issues, it still has some limitations, as shown in the following example:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "id": "b21013ec-b0e4-4428-9bb1-e2f309df563a",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2025-01-15T06:51:02.890690Z",
          "start_time": "2025-01-15T06:51:00.165152Z"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Yes, a Content Development Engineer is responsible for designing and developing high-quality educational materials and courses, including writing syllabi, creating courseware, and designing assessment tools."
          ]
        }
      ],
      "source": [
        "rag.ask('Please help me review this sentence: Does a technical content engineer need to design and develop high-quality educational materials and courses?', query_engine=query_engine)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "25dd1116-048c-4599-ba9d-98750978504a",
      "metadata": {},
      "source": [
        "From the experimental results above, it's clear that the Q&A robot responded directly to the user's statement without recognizing the intended task of reviewing a sentence for errors. To understand why this occurred, let's look at what content the Q&A bot retrieved from the knowledge base before generating its response.\n",
        "\n",
        "The following excerpt  shows part of the retrieved content:\n",
        "\n",
        "```\n",
        "Core Responsibilities\n",
        "Integrating educational theory with technical practice, support learners' growth and development through high-quality content creation.\n",
        "```\n",
        "\n",
        "You can refer to <2_4_Automate_Evaluation_of_FAQ_Bot_Performance> to learn how to view the specificfragments retrieved  from the knowledge base.\n",
        "\n",
        "As seen from the retrieved content, most of the information relates to the responsibilities of a technical content engineer. This context likely influenced the Q&A bot's response, causing it to focus on content creation or editing rather than performing the requested document review.\n",
        "\n",
        "Since contextual information greatly impacts the responses generated by LLMs, it is not always necessary to use the RAG method for every query. You can leverage LLMs to first perform intent recognitionclassifying the type of user question.\n",
        "\n",
        "For example:\n",
        "\n",
        "- If the task involves document review, grammar checking, or content translation, the input should bypass the RAG pipeline and be handled directly by the LLM.\n",
        "- Only for internal knowledge queries—such as \"What are the guidelines for writing API documentation?\"—should the RAG pipeline be used to generate answers based on the knowledge base."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5df6389a-cfa2-450e-b7fe-72e24dd79838",
      "metadata": {},
      "source": [
        "<img src=\"https://img.alicdn.com/imgextra/i3/O1CN01nWw7IO2AGCuuujq8G_!!6000000008175-2-tps-3329-1398.png\" width=\"1000\">  \n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f9fe0489-872c-466b-855a-3768a5127f9e",
      "metadata": {},
      "source": [
        "There are two main methods for using lLLMs to perform intent recognition:\n",
        "\n",
        "- Using prompts: By designing specific prompts, you can guide the LLM to generate responses that meet your expectations. This method does not require modifying the model’s parameters; instead it relies on well-constructed inputs to activate knowledge already embedded within the model.\n",
        "- Fine-tuning the model: Based on a pre-training foundational model, you can further train the model using specific annotated data to improve its ability to classify intents. This process involves adjusting some or all of the model’s parameters to better suit your use case.\n",
        "\n",
        "In this section, we will focus on helping the LLM perform intent recognition by crafting effective prompts. You can explore the method of fine-tuning in the subsection <2_7_Improve_Model_Accuracy_and_Efficiency_via_Fine_Tuning>.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fe46f635-ac13-488d-9ab7-89b49ae8878d",
      "metadata": {},
      "source": [
        "### 5.1 Intent recognition\n",
        "\n",
        "Next, we will construct prompts to enable LLMs to classify questions. Since many applications require structured output after intent recognition—such as for document review or integration with RAG pipelines—the following prompting techniques will be used to ensure accurate classification of user queries:\n",
        "\n",
        "* Specify the output format: Clearly define the output format to make classification results standardized and easy to parse.\n",
        "* Few-shot examples: Include examples to help the LLM understand the characteristics and classification rules of each category."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "id": "63ff599e-a7fa-41d8-b734-d949e9f93900",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2025-01-15T06:51:10.939753Z",
          "start_time": "2025-01-15T06:51:09.207243Z"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Document Review \n",
            "\n",
            "Content Translation \n",
            "\n",
            "Internal Company Document Query \n",
            "\n"
          ]
        }
      ],
      "source": [
        "from chatbot import llm\n",
        "\n",
        "# Build the prompt\n",
        "prompt = '''\n",
        "[Role Background]\n",
        "You are a question classification router, responsible for determining the type of user questions and categorizing them into one of the following three categories:\n",
        "1. Internal Company Document Query\n",
        "2. Content Translation\n",
        "3. Document Review\n",
        "\n",
        "[Task Requirements]\n",
        "Your task is to judge the intent based on the user's input content and select only the most appropriate category. Output only the category name without additional explanation. The judgment criteria are as follows:\n",
        "\n",
        "- If the question involves company policies, processes, internal tools, or job descriptions and responsibilities, choose \"Internal Company Document Query\".\n",
        "- If the input contains any foreign language or words like \"translation\", choose \"Content Translation\".\n",
        "- If the question involves checking or summarizing external documents or link content, choose \"Document Review\".\n",
        "- The user's previous inputs have no relation to the question classification; consider each dialogue independently for classification.\n",
        "\n",
        "[Few-shot Examples]\n",
        "Example 1: User input: \"What are the commonly used project management tools within our company?\"\n",
        "Category: Internal Company Document Query\n",
        "\n",
        "Example 2: User input: \"Please translate the following sentence into French: How can we finish the assignment on time?\"\n",
        "Category: Content Translation\n",
        "\n",
        "Example 3: User input: \"Please review the document at this link: https://help.aliyun.com/zh/model-studio/user-guide/long-context-qwen-long\"\n",
        "Category: Document Review\n",
        "\n",
        "Example 4: User input: \"Please review the following content: Does a technical content engineer need to design and develop high-quality educational materials and courses?\"\n",
        "Category: Document Review\n",
        "\n",
        "Example 5: User input: \"What are the core responsibilities of a technical content engineer?\"\n",
        "Category: Internal Company Document Query\n",
        "\n",
        "[User Input]\n",
        "The following is the user's input, please determine the classification:\n",
        "'''\n",
        "\n",
        "# Get the type of question\n",
        "def get_question_type(question):\n",
        "    return llm.invoke(prompt + question)\n",
        "\n",
        "print(get_question_type('https://www.promptingguide.ai/zh/techniques/fewshot'), '\\n')\n",
        "print(get_question_type('Translate the following sentence into German: That is a big one I dont know why'), '\\n')\n",
        "print(get_question_type('As a technical content engineer, what should I pay attention to?'), '\\n')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1f74506a-504d-47a2-bb9d-e17198294445",
      "metadata": {},
      "source": [
        "By providing clear output formats and few-shot examples, the Q&A bot can more accurately identify question types and produce responses in the expected format. This optimization makes classification tasks more standardized and lays the foundation for incorporating intent recognition into the Q&A bot."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fb001699-4451-4e1f-808d-1b38a80c9431",
      "metadata": {},
      "source": [
        "### 5.2 Applying intent recognition to Q&A bots\n",
        "\n",
        "Once the Q&A bot can recognize the intent of a user's question, it can first classify the type of query, then use tailored prompts and workflows to generate the most appropriate response.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "id": "c5d8de14-53ee-4d5f-8f5a-0dfd9bf4e0ba",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2025-01-15T06:51:13.272321Z",
          "start_time": "2025-01-15T06:51:13.111875Z"
        }
      },
      "outputs": [],
      "source": [
        "def ask_llm_route(question):\n",
        "    question_type = get_question_type(question)\n",
        "    print(f'Question: {question}\\nType: {question_type}')\n",
        "  \n",
        "    reviewer_prompt = \"\"\"\n",
        "    [Role Background]\n",
        "    You are a document error-checking expert responsible for identifying obvious errors in documents or web content.\n",
        "    [Task Requirements]\n",
        "    - Your response should be concise.\n",
        "    - If there are no obvious issues, reply directly with 'No issues'.\\n\n",
        "    [Input as follows]\\n\"\"\"\n",
        "  \n",
        "    translator_prompt = \"\"\"\n",
        "   [Task Requirements]\n",
        "    You are a translation expert who identifies text in different languages and translates it into the target language. If not specified, the target language is French.\n",
        "    [Input as follows]\\n\"\"\"\n",
        "\n",
        "    if question_type == 'Document Review':\n",
        "        return llm.invoke(reviewer_prompt + question)\n",
        "    elif question_type == 'Internal Company Document Query':\n",
        "        return rag.ask(question, query_engine=query_engine)\n",
        "    elif question_type == 'Content Translation':\n",
        "        return llm.invoke(translator_prompt + question)\n",
        "    else:\n",
        "        return \"Unable to recognize the question type. Please re-enter.\"\n",
        "\n",
        "query_engine =rag.create_query_engine(index=rag.load_index())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "id": "d5d6ec83-8a99-45a5-a305-5088adc89be0",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2025-01-15T06:51:22.925058Z",
          "start_time": "2025-01-15T06:51:14.191174Z"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Question: https://www.promptingguide.ai/zh/techniques/fewshot\n",
            "Type: Document Review\n",
            "No issues \n",
            "\n",
            "Question: Please help me check this document: Do technicel content engineers need to optimize and update content and collaborate across departments?\n",
            "Type: Document Review\n",
            "The word \"technicel\" is misspelled. It should be \"technical.\" \n",
            "\n",
            "Question: Do technical content engineers need to optimize and update content and collaborate across departments?\n",
            "Type: Internal Company Document Query\n",
            "Yes, technical content engineers are responsible for optimizing and updating content to ensure it remains relevant and effective. They also collaborate across departments, working closely with instructional designers, educational psychologists, technical teams, and marketing personnel to ensure the smooth implementation of technical processes and effective communication to the target audience.None \n",
            "\n",
            "Question: Translate the following sentence into French: A true master always carries the heart of a student.\n",
            "Type: Content Translation\n",
            "Un vrai maître porte toujours le cœur d'un étudiant. \n",
            "\n"
          ]
        }
      ],
      "source": [
        "# Question 1\n",
        "print(ask_llm_route('https://www.promptingguide.ai/zh/techniques/fewshot'), '\\n')\n",
        "\n",
        "# Question 2\n",
        "print(ask_llm_route('Please help me check this document: Do technicel content engineers need to optimize and update content and collaborate across departments?'), '\\n')\n",
        "\n",
        "# Question 3\n",
        "print(ask_llm_route('Do technical content engineers need to optimize and update content and collaborate across departments?'), '\\n')\n",
        "\n",
        "# Question 4:\n",
        "print(ask_llm_route('Translate the following sentence into French: A true master always carries the heart of a student.'), '\\n')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "d6a95edd-ca52-434f-b6b5-6b0537fdfec6",
      "metadata": {},
      "source": [
        "As shown in the above experiments, the optimized Q&A bot does not always go through the RAG pipeline for every question. This approach not only saves computational resources but also prevents the knowledge base from interfering with the model's reasoning process, which could otherwise lead to lower-quality or irrelevant responses.\n",
        "\n",
        "- Resource savings: For tasks such as checking document errors, the LLM can respond without needing to retrieve reference materials. This highlights that previous implementations may have unnecessarily consumed resources by always using the RAG pipeline.\n",
        "- Avoiding misunderstandings: In earlier versions, the system retrieved reference materials for every query. These recalled text segments could sometimes confuse the LLM, affecting its ability to accurately understand the question and leading to irrelevant or off-target responses."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c7c107a1",
      "metadata": {},
      "source": [
        "## 6. Reasoning LLMs\n",
        "\n",
        "The prompt techniques and prompt frameworks discussed earlier are widely applicable to general LLMs, such as Qwen2.5-max, GPT-4, and DeepSeek-V3. These models are designed for a broad range of tasks, including general conversation, knowledge-based Q&A, and text generation. \n",
        "\n",
        "In addition to general-purpose LLMs, there is another category of models specifically designed for reasoning—Reasoning LLMs. These models excel in tasks that require logical thinking, multi-step problem-solving, and mathematical computation.\n",
        "\n",
        "### 6.1 What Are reasoning LLMs?\n",
        "\n",
        "You may have already learned about Alibaba Cloud's reasoning LLM [QwQ](https://help.aliyun.com/zh/model-studio/user-guide/qwq), through the extended reading in Section 2.1. To make it easier to use in this section, the code from Section 2.1 has been updated. Try running the following example:\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "id": "500b6e30",
      "metadata": {},
      "outputs": [],
      "source": [
        "from openai import OpenAI\n",
        "import os\n",
        "\n",
        "def reasoning_model_response(user_prompt, system_prompt=\"You are a programming assistant.\", model=\"qwq-32b\"):\n",
        "    \"\"\"\n",
        "    prompt: The prompt input by the user\n",
        "    model: Here we use qwq-32b as an example. You can replace it with other inference model names as needed, such as: deepseek-r1\n",
        "    \"\"\"\n",
        "    # Initialize client\n",
        "    client = OpenAI(\n",
        "        api_key=os.getenv(\"DASHSCOPE_API_KEY\"),\n",
        "        base_url=\"https://dashscope.aliyuncs.com/compatible-mode/v1\"\n",
        "    )\n",
        "\n",
        "    # Initialize status variables\n",
        "    is_answering = False\n",
        "\n",
        "    # Initiate streaming request\n",
        "    completion = client.chat.completions.create(\n",
        "        model=model,\n",
        "        # messages=[{\"role\": \"user\", \"content\": prompt}],\n",
        "        messages=[\n",
        "            {\"role\": \"system\", \"content\": system_prompt},\n",
        "            {\"role\": \"user\", \"content\": user_prompt}\n",
        "        ],\n",
        "        stream=True,\n",
        "    )\n",
        "\n",
        "    # Print thinking process title\n",
        "    print(\"\\n\" + \"=\" * 20 + \"Thinking Process\" + \"=\" * 20 + \"\\n\")\n",
        "\n",
        "    # Handle streaming response\n",
        "    for chunk in completion:\n",
        "        if chunk.choices:\n",
        "            delta = chunk.choices[0].delta\n",
        "            if hasattr(delta, 'reasoning_content') and delta.reasoning_content is not None:\n",
        "                # Process thinking process content\n",
        "                print(delta.reasoning_content, end='', flush=True)\n",
        "            else:\n",
        "                # Switch to answer output mode\n",
        "                if delta.content != \"\" and not is_answering:\n",
        "                    print(\"\\n\" + \"=\" * 20 + \"Complete Response\" + \"=\" * 20 + \"\\n\")\n",
        "                    is_answering = True\n",
        "                # Process answer content\n",
        "                if delta.content:\n",
        "                    print(delta.content, end='', flush=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "id": "9e65df7d",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "====================Thinking Process====================\n",
            "\n",
            "Okay, the user asked, \"Who are you?\" Let me think about how to respond.\n",
            "\n",
            "First, I need to remember my role. I'm a programming assistant, right? But the user might not know that yet. So I should start by introducing myself as Qwen, a large language model developed by Alibaba Cloud. That's the basic info.\n",
            "\n",
            "Wait, but the user specifically mentioned being a programming assistant. Maybe I should highlight that aspect more. They might be looking for help with coding. So I should mention my capabilities in programming, like writing code, explaining concepts, debugging, etc.\n",
            "\n",
            "Hmm, also, I should keep it friendly and open-ended. Let them know I can help with various tasks beyond just programming. Maybe list some examples to give them an idea of what I can do. But not too long, keep it concise.\n",
            "\n",
            "Oh, and I should make sure to invite them to ask for help. Something like \"Feel free to ask...\" to encourage interaction. Alright, putting it all together now.\n",
            "====================Complete Response====================\n",
            "\n",
            "Hello, I’m Qwen, a large language model developed by Alibaba Cloud. I'm designed to assist with a wide range of tasks, including coding, writing, problem-solving, and more. I can help you write code in various programming languages, explain technical concepts, debug errors, and even brainstorm ideas for projects. Whether you need help with a specific programming problem, want to learn a new concept, or just have a question, feel free to ask! How can I assist you today?"
          ]
        }
      ],
      "source": [
        "reasoning_model_response(user_prompt=\"Who are you?\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3fea9f93",
      "metadata": {},
      "source": [
        "As shown in the example, reasoning LLMs include an additional \"thinking process\" step, which mimics how humans approach complex problems. This allows the model to break down the problem logically before providing a final answer. This structured approach helps reduce errors and improves transparency in the model’s decision-making.\n",
        "\n",
        "During the thinking process, if a contradiction is found, the model can go back and revise its logic, making it easier for users to understand and verify the reasoning path. This feature is especially useful in scenarios requiring rigorous logic, such as mathematical problem-solving, legal analysis, or scientific reasoning.\n",
        "\n",
        "Compared with general LLMs, reasoning models are usually more reliable when solving complex problems, such as mathematical problem-solving, code writing, legal case analysis, and other scenarios that require rigorous reasoning. This does not mean that reasoning models are necessarily better; both types of models have their own application scenarios. The following table compares these two types of models across several key dimensions:\n",
        "\n",
        "| Dimension            | Reasoning Model  | General Model  |\n",
        "|-------------------|------------------|------------|\n",
        "| Design Goal     | Focuses on tasks requiring deep analysis such as **logical reasoning, multi-step problem solving, and mathematical calculations** | Targets broad scenarios such as **general conversation, knowledge-based Q&A, text generation |\n",
        "| Training Data Emphasis  | Enhanced reasoning capabilities through large datasets of **math problem solutions, code logic, scientific reasoning**, etc. | Covers massive data across multiple domains such as **encyclopedias, literature, conversations**, etc. |\n",
        "| Typical Output Characteristics | Outputs include **complete derivations**, focusing on the integrity of logical chains | Outputs are **concise and direct**, emphasizing natural language expression of results |\n",
        "| Response Speed | Complex reasoning tasks result in **slower responses** (requires multi-step calculations) | Routine tasks have **faster responses** (mainly single-step generation) |\n",
        "\n",
        "This comparison highlights that reasoning models excel in structured, analytical tasks, while general LLMs are more versatile for everyday use cases. Choosing the right model depends on the specific requirements of the task at hand. \n",
        "\n",
        "Here are some tips:\n",
        "* **Clearly defined general tasks**: For problems that are well-defined and straightforward, general models typically perform well.\n",
        "* **Complex tasks**: For highly complex tasks that require  **more precise and reliable** answers,  **reasoning models** are recommended. These tasks may include:\n",
        "    * Ambiguous tasks: When there is very little task-related information available, and clear guidance cannot be provided to the model.\n",
        "    * Finding a needle in a haystack: When dealing with large amounts of unstructured data, extracting the most relevant information, or identifying connections and differences.\n",
        "    * Debugging and improving code: When the task involves reviewing and refining large volumes of code.\n",
        "* **Speed and cost**: Reasoning models generally take longer to process due to their detailed reasoning steps. If you time and cost are critical factors and the task complexity is low, **general models** may be a more efficient choice.\n",
        "\n",
        "In addition, you can combine both types of models in your application: use the reasoning model for planning and decision-making, and the general model for executing specific tasks."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "580c0b84",
      "metadata": {},
      "source": [
        "### 6.2 Prompt techniques suitable for reasoning LLMs\n",
        "\n",
        "Reasoning models can provide detailed and well-structured responses even when faced with relatively ambiguous tasks. You can still ensure the baseline quality of reasoning in large language models through **prompt techniques**:<br>\n",
        "\n",
        "#### Technique One: Keep task prompts concise and clear, providing sufficient background information\n",
        "\n",
        "The **clear expression of requirements** introduced in Section 4.1 also applies to reasoning models. Although these models are highly capable, they cannot \"read minds.\" Therefore, it's important to keep your prompts **concise and clear**, allowing the reasoning model to focus on the core task. Providing enough context ensures that the model understands the problem and can generate accurate, logical responses.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "id": "fe5cc8ad",
      "metadata": {},
      "outputs": [],
      "source": [
        "def example(a):\n",
        "  b = []\n",
        "  for i in range(len(a)):\n",
        "    b.append(a[i]*2)\n",
        "  return sum(b)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "61e6f145",
      "metadata": {},
      "source": [
        "Through the above example, you can see that even if you only provide a large inference model with a piece of code, it can still generate detiled and thoughtful responses through a series of reasoning steps. However, the generated reasoning may contain a lot of information that is not directly relevant to your needs. To get more focused and targeted results, it's important to clearly define the **task objective** and guide the model toward the specific outcome you want. Here's an example:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "id": "10c90bd3",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "====================Thinking Process====================\n",
            "\n",
            "Okay, let me try to figure out what's wrong with this Python code and how to optimize it. The function example(a) takes a list 'a', creates a new list 'b' by doubling each element, then returns the sum of 'b'. \n",
            "\n",
            "Hmm, first, I should check if there are any obvious issues. The code seems to work, but maybe there's a more efficient way. Let me think step by step.\n",
            "\n",
            "First, the code uses a for loop with range(len(a)), which is common but not the most Pythonic way. Maybe using a list comprehension would be better here. That would make the code shorter and possibly faster since list comprehensions are optimized in Python.\n",
            "\n",
            "Wait, but even better, maybe we can avoid creating the list 'b' altogether. Since the final result is the sum of all elements multiplied by 2, instead of creating a list and then summing, we can just iterate through each element, multiply by 2, and add to a running total. That way, we don't need to store all the doubled elements in memory. That would save space, especially for large lists.\n",
            "\n",
            "Let me see. The original code does:\n",
            "\n",
            "b = [a[0]*2, a[1]*2, ...], then sum(b). So the sum is sum(a[i]*2 for all i). Which is the same as 2 * sum(a). Wait a minute! Oh! Because summing each element multiplied by 2 is the same as multiplying the sum of the original list by 2. That's a mathematical simplification. So instead of looping through each element, doubling, and summing, you can just compute 2 * sum(a). That would be way more efficient, O(1) time instead of O(n). That's a huge optimization!\n",
            "\n",
            "So the problem with the original code is that it's using a loop and a list when it's unnecessary. The optimized version would just do 2 * sum(a). That's a much better approach.\n",
            "\n",
            "Wait, but let me make sure that's correct. Suppose the list is [1,2,3], original code would compute [2,4,6], sum is 12. 2*sum([1,2,3]) is 2*6=12. Yes, that's the same. So the mathematical approach works here.\n",
            "\n",
            "Therefore, the main issue is that the code is using a loop and temporary list when a simple mathematical formula can achieve the same result more efficiently. The optimized code would just return 2 * sum(a).\n",
            "\n",
            "Another possible optimization is using a generator expression instead of building a list. Even if someone didn't realize the math trick, they could replace the loop with sum(x*2 for x in a), which avoids creating the list 'b' and is more concise. So that's another way to improve it, but the math approach is even better.\n",
            "\n",
            "So the answer should point out both the unnecessary loop and list, and suggest either the generator expression or the mathematical simplification. The best optimization is the math one.\n",
            "====================Complete Response====================\n",
            "\n",
            "The code works but can be optimized for both readability and efficiency. Here's the breakdown:\n",
            "\n",
            "### Issues:\n",
            "1. **Unnecessary List Creation**: The code creates an intermediate list `b` to store doubled values, which is memory inefficient, especially for large inputs.\n",
            "2. **Suboptimal Time Complexity**: The loop runs in O(n) time, but the problem can be solved in O(1) time with a mathematical insight.\n",
            "\n",
            "### Optimized Solutions:\n",
            "\n",
            "#### 1. **Mathematical Simplification (Best Approach)**\n",
            "   The sum of all elements multiplied by 2 is equivalent to twice the sum of the original list. This reduces time complexity to O(1) and eliminates the need for loops or lists.\n",
            "\n",
            "```python\n",
            "def example(a):\n",
            "    return 2 * sum(a)\n",
            "```\n",
            "\n",
            "#### 2. **Using a Generator Expression (Alternative Approach)**\n",
            "   If the mathematical simplification isn't obvious, you can use a generator expression to avoid creating an intermediate list:\n",
            "\n",
            "```python\n",
            "def example(a):\n",
            "    return sum(x * 2 for x in a)\n",
            "```\n",
            "\n",
            "### Key Improvements:\n",
            "- **Memory Efficiency**: Avoids storing the doubled values in a list.\n",
            "- **Readability**: More concise and Pythonic.\n",
            "- **Performance**: The mathematical approach runs in constant time, while the generator expression still runs in linear time but is faster and cleaner than the original loop.\n",
            "\n",
            "### Example Usage:\n",
            "```python\n",
            "print(example([1, 2, 3]))  # Output: 12 (since 2*(1+2+3) = 12)\n",
            "```\n",
            "\n",
            "The best optimization is the mathematical approach, which is both the simplest and most efficient."
          ]
        }
      ],
      "source": [
        "prompt_A = \"\"\"\n",
        "What's wrong with the following Python code? How to optimize it?\n",
        "def example(a):\n",
        "    b = []\n",
        "    for i in range(len(a)):\n",
        "        b.append(a[i]*2)\n",
        "    return sum(b)\n",
        "\"\"\"\n",
        "\n",
        "reasoning_model_response(user_prompt=prompt_A)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e5500ab0",
      "metadata": {},
      "source": [
        "Similarly, you can further refine the scope by combining techniques from this section, such as **4.2 Limiting roles and audience** and **4.3 Specifying output formats**, to ensure the results align with your expectations.Additionally, when the prompt is relatively complex, using delimiters can help the model better understand your intent and focus on the key elements of the task."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "id": "b0d81507",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "====================Thinking Process====================\n",
            "\n",
            "Okay, let's see. The user wants me to optimize this Python function. The function takes a list 'a', creates a new list 'b' where each element is doubled, then returns the sum of 'b'. Hmm, right now it's using a for loop with append and then summing. I need to think of ways to make this faster or more efficient.\n",
            "\n",
            "First, maybe using a list comprehension instead of a for loop. List comprehensions are generally faster in Python. So instead of initializing an empty list and appending each time, just do [x*2 for x in a]. That should be quicker.\n",
            "\n",
            "Wait, but then you still have to sum the list. Oh right, but even better, maybe combine the doubling and summing in a generator expression to avoid creating the list at all. Because sum can take a generator, so you don't need to store all elements in a list first. That would save memory and time, especially for large lists. So instead of building the list 'b', just compute each element on the fly and sum them directly. So like sum(x*2 for x in a). That's better.\n",
            "\n",
            "Another thought: using the built-in map function with a lambda. Like sum(map(lambda x: x*2, a)). I think map is also optimized and might be faster than a list comp in some cases, but maybe not as much as the generator. Or maybe the generator is better.\n",
            "\n",
            "Wait, but in the original code, the list 'b' is necessary? No, because the sum only needs the values once. So using a generator expression would eliminate the need to create the list, which is better. So that's probably the best optimization here. So the first optimization is replacing the loop with a list comp, but the second is even better by using a generator expression directly in sum.\n",
            "\n",
            "So the first optimization solution would be using a list comprehension, which is a common beginner mistake to use a for loop with append. The second solution would be using a generator expression to avoid creating the intermediate list entirely. That's probably the main two optimizations here.\n",
            "\n",
            "Let me check. The original code has O(n) time and O(n) space. The first optimization (list comp) is still O(n) space. The second (generator) reduces space to O(1) (well, O(1) extra space besides the input), so better. So that's a better optimization.\n",
            "\n",
            "So the problem description for the first solution is using a slow for loop with append, which can be replaced by a list comprehension. The second solution is using a generator expression to eliminate the list.\n",
            "\n",
            "Wait, but maybe there's another way? Like using the sum multiplied by 2. Wait, because sum(a) * 2 is equivalent to summing each element doubled. Oh! Wait a second, that's a mathematical optimization. Because each element is multiplied by 2 and summed, that's the same as 2 times the sum of the original list. So instead of doing sum([x*2 for x in a]), you can just do 2 * sum(a). That would be a huge optimization, both in time and space, because you don't have to loop through the list twice (once to double each element and again to sum?), no, actually in the original code, the loop is for building the list, then sum is another pass? Wait, no. Wait the original code loops through the list once to build the list, then sum is another loop over the list. Wait, no: the original code's loop is O(n) to build the list, then sum is another O(n) pass. So total O(2n) time. Whereas if you just do sum(a) *2, that's O(n) time, which is better. Oh! That's a big optimization. So that's probably the best one.\n",
            "\n",
            "Wait, the original code is:\n",
            "\n",
            "for i in range(len(a)): loop over each element, appends a[i]*2. So building the list is O(n) time, then sum is another O(n). So total O(n) steps. Wait, but actually sum is another O(n). So total O(2n) which is O(n). But the sum(a)*2 approach would be O(n) as well, but with half the operations. So that's a better optimization. So that's actually a mathematical optimization, not just code structure.\n",
            "\n",
            "Wait, the problem says \"function performance optimization, optimize the code in the code\". So perhaps that's the best solution here. So that's the third optimization.\n",
            "\n",
            "Hmm, so maybe the user is expecting that. Let me think again. The original function is:\n",
            "\n",
            "def example(a):\n",
            "    b = []\n",
            "    for i in range(len(a)):\n",
            "        b.append(a[i]*2)\n",
            "    return sum(b)\n",
            "\n",
            "The sum(b) is O(n) time. The loop is O(n) time. So total O(n) time. But the space is O(n) because of the list b. The mathematical approach of 2*sum(a) would take O(n) time and O(1) space, which is better. So that's a better optimization.\n",
            "\n",
            "So the problem here is that the original code is unnecessarily creating a list and looping through it twice (once to build the list, then again to sum it). Wait, no, the loop builds the list and then sum(b) loops through the list again. So the total operations are 2n (n for the loop, n for the sum). Whereas the mathematical approach is just n operations (sum the list once, multiply by 2). So that's a better optimization.\n",
            "\n",
            "Therefore, the best optimization is to replace the entire function with return 2 * sum(a). That's a much better optimization. But would a beginner think of that? The task is to optimize the code, so maybe that's the main point here.\n",
            "\n",
            "Alternatively, maybe the user wants code optimizations without changing the logic. Wait, but the problem says \"optimize the code\", so changing the logic is okay as long as it's correct. Since 2*sum(a) is the same as sum each element doubled, that's correct.\n",
            "\n",
            "So that's probably the best optimization. So I need to present that as one of the solutions.\n",
            "\n",
            "So possible solutions:\n",
            "\n",
            "1. Replace the for loop with a list comprehension, which is faster than appending in a loop. So the code becomes:\n",
            "\n",
            "def example(a):\n",
            "    b = [x*2 for x in a]\n",
            "    return sum(b)\n",
            "\n",
            "That's better because list comprehensions are more optimized. But still uses O(n) space.\n",
            "\n",
            "2. Use a generator expression to eliminate the list. So:\n",
            "\n",
            "def example(a):\n",
            "    return sum(x*2 for x in a)\n",
            "\n",
            "This is better because it doesn't create the list, just iterates once. So O(n) time, O(1) space (except for the input). That's better than the first solution.\n",
            "\n",
            "3. The mathematical approach: return 2 * sum(a). That's even better, same time, less operations, and no extra space.\n",
            "\n",
            "So these are three possible optimizations. The user's format says if there are multiple solutions, output them in order. So I should list all three?\n",
            "\n",
            "Wait, the problem says \"if there are multiple optimization solutions\". The user's code has a few inefficiencies. The loop using range(len(a)) and a[i] is less Pythonic than iterating over elements directly. Also, building a list and then summing it is unnecessary.\n",
            "\n",
            "So the first optimization could be using a list comprehension. The second using a generator. The third using the mathematical shortcut.\n",
            "\n",
            "Therefore, I should present these as three solutions. Let's structure them.\n",
            "\n",
            "First, the problem with the loop and append. Solution: replace with list comprehension.\n",
            "\n",
            "Second problem: creating a list when it's not needed. Solution: use generator expression.\n",
            "\n",
            "Third problem: unnecessary computation, can be simplified mathematically. Solution: 2*sum(a).\n",
            "\n",
            "So I need to present each as a separate optimization solution. The user's code example may have all these possibilities.\n",
            "\n",
            "So the first solution would be:\n",
            "\n",
            "Problem: Using a for loop with append is slower than a list comprehension.\n",
            "\n",
            "Second solution: Using a generator expression to avoid creating the list.\n",
            "\n",
            "Third solution: Mathematical simplification to avoid looping twice and creating lists.\n",
            "\n",
            "So I'll present these three as three optimization solutions. The user might expect all three. Let me check the code again.\n",
            "\n",
            "Original code uses a loop with range(len(a)), which is a common beginner mistake. A better way is to loop over the elements directly. So in the first solution, the list comprehension would do that.\n",
            "\n",
            "So the first optimization solution would be:\n",
            "\n",
            "Problem Description: The original code uses a for loop with append, which is slower than list comprehensions. \n",
            "\n",
            "Optimization Solution: Replace the loop with a list comprehension.\n",
            "\n",
            "Example code:\n",
            "\n",
            "def example(a):\n",
            "    return sum([x*2 for x in a])\n",
            "\n",
            "Then the second optimization would be using the generator expression, avoiding the list.\n",
            "\n",
            "Problem Description: Creating an intermediate list is unnecessary and consumes memory. \n",
            "\n",
            "Solution: Use a generator expression to compute elements on the fly.\n",
            "\n",
            "Example code:\n",
            "\n",
            "def example(a):\n",
            "    return sum(x*2 for x in a)\n",
            "\n",
            "Third solution is the mathematical approach.\n",
            "\n",
            "Problem Description: The function can be simplified using mathematical operations to avoid unnecessary loops and list creation.\n",
            "\n",
            "Solution: Multiply the sum of the original list by 2.\n",
            "\n",
            "Example code:\n",
            "\n",
            "def example(a):\n",
            "    return 2 * sum(a)\n",
            "\n",
            "These are three valid optimizations. The third one is the most optimal in terms of both time and space.\n",
            "\n",
            "So I'll structure them as three solutions.\n",
            "====================Complete Response====================\n",
            "\n",
            "[Optimization Solution 1]\n",
            "Problem Description: The original code uses a for loop with `append()` to create a list, which is slower and less Pythonic than list comprehensions.\n",
            "Optimization Solution: Replace the loop with a list comprehension for faster execution and cleaner syntax.\n",
            "Example Code:\n",
            "```python\n",
            "def example(a):\n",
            "    return sum([x * 2 for x in a])\n",
            "```\n",
            "\n",
            "[Optimization Solution 2]\n",
            "Problem Description: Creating an intermediate list (e.g., `b`) wastes memory and time when it is not necessary.\n",
            "Optimization Solution: Use a generator expression to compute elements on the fly without storing them in a list.\n",
            "Example Code:\n",
            "```python\n",
            "def example(a):\n",
            "    return sum(x * 2 for x in a)\n",
            "```\n",
            "\n",
            "[Optimization Solution 3]\n",
            "Problem Description: The function redundantly doubles each element and sums them, which can be simplified mathematically.\n",
            "Optimization Solution: Use the mathematical property that doubling each element and summing is equivalent to doubling the sum of the original list.\n",
            "Example Code:\n",
            "```python\n",
            "def example(a):\n",
            "    return 2 * sum(a)\n",
            "```"
          ]
        }
      ],
      "source": [
        "prompt_B = \"\"\"\n",
        "<audience>Beginner Python developers</audience>\n",
        "\n",
        "<task>Function performance optimization, optimize the code in the code.</task>\n",
        "\n",
        "<format>\n",
        "If there are multiple optimization solutions, please output them in the following format:\n",
        "[Optimization Solution X]\n",
        "Problem Description: [Description]\n",
        "Optimization Solution: [Description]\n",
        "Example Code: [Code Block]\n",
        "</format>\n",
        "\n",
        "<code>\n",
        "def example(a):\n",
        "  b = []\n",
        "  for i in range(len(a)):\n",
        "    b.append(a[i]*2)\n",
        "  return sum(b)\n",
        "</code>\n",
        "\"\"\"\n",
        "\n",
        "reasoning_model_response(user_prompt=prompt_B)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c163ae4c",
      "metadata": {},
      "source": [
        "#### Technique two: Avoid CoT prompts\n",
        "In Section 4.5, you learned how to improve response quality by using the chain-of-thought (CoT)technique with LLMs. However, for reasoning models, it is generally unnecessary to prompt them with phrases like \"think step by step\" or \"explain your reasoning.\" These models are designed to perform deep thinking naturally. Adding such instructions may actually limit their performance unless you specifically need the model to follow a strict reasoning path, which is uncommon.\n",
        "\n",
        "\n",
        "#### Technique three: Adjust prompts based on model responses\n",
        "Reasoning models are well-suited for analyzing their own thought processes, because their responses often include **reasoning steps**. This makes it easier for you to refine prompts based on the model's output. You don’t need to worry about whether your initial prompt is perfect—simply continue engaging with the model, providing additional information, and refining your prompts during the conversation.\n",
        "\n",
        "For example, if your descriptions are too abstract or not accurately described, you can use the technique of adding examples, as discussed in Section 4.4, to clarify the task. These examples can often be drawn from previous interactions with the model.This iterative process can be repeated multiple times, continuously adjusting prompts and allowing the model to refine its reasoning until it meets your expectations.\n",
        "\n",
        "#### Technique four: Collaborative task completion between reasoning models and general models\n",
        "Reasoning models act like \"super-smart colleagues,\" effectively assisting you with tasks like reasoning and planning. However, for execution tasks involving repetitive actions, reasoning models may overthink, which could be inefficient.\n",
        "\n",
        "A better approach is to let reasoning models and general models work together, leveraging their respective strengths:\n",
        "* Reasoning models handle \"slow thinking\" tasks such as planning, analysis, and problem-solving.\n",
        "* General models focus on \"fast thinking\" tasks, such as executing specific actions or generating content quickly.\n",
        "\n",
        "Try running and understanding the following project, which demonstrates how these models can collaborate:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "01807cca",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "\n",
            "=== [Planning Thinking Phase] Using reasoning model: qwq-32b===\n",
            "Okay, the user wants me to schedule next week's product launch and create related preparation tasks. Let me break this down.\n",
            "\n",
            "First, I need to use the available tools. The tools are create_task and schedule_event. The schedule_event requires a title and a datetime. The create_task needs a description and priority.\n",
            "\n",
            "Starting with scheduling the event. The product launch is next week. I should pick a specific day and time. Let's say next Friday at 2 PM. So the title could be \"Product Launch Event\" and the time would be something like \"2023-11-17T14:00:00\" assuming next Friday is the 17th. Wait, but the exact date might depend on the current date. Since the user didn't specify, I'll use a placeholder or example date.\n",
            "\n",
            "Next, the preparation tasks. What tasks are usually needed before a product launch? Maybe finalizing the product features, preparing marketing materials, setting up the event venue, and training the team. Each of these should be a task with a priority. Let's assign priorities: maybe finalizing features is high (priority 1), marketing materials also 1, venue setup maybe 2, and training could be 2. Or maybe all high? The user might want the most critical first. I'll go with 1 for the first two and 2 for the others.\n",
            "\n",
            "So the steps would be first schedule the event with the title and time. Then create each task with their descriptions and priorities. Need to make sure each step uses the correct tool name and parameters with examples. Let me structure each step as a list item in markdown, indicating the tool and parameters with example values.\n",
            "\n",
            "=== [Planning Generation Phase] Using reasoning model: qwq-32b===\n",
            "1. **Schedule the product launch event**:  \n",
            "   - Tool: [[schedule_event]]  \n",
            "   - Parameters:  \n",
            "     `title`: \"Product Launch Event\"  \n",
            "     `time`: \"2023-11-17T14:00:00\" (e.g., next Friday at 2:00 PM)  \n",
            "\n",
            "2. **Create task: Finalize product features**:  \n",
            "   - Tool: [[create_task]]  \n",
            "   - Parameters:  \n",
            "     `description`: \"Review and finalize all product features for launch\"  \n",
            "     `priority`: 1  \n",
            "\n",
            "3. **Create task: Prepare marketing materials**:  \n",
            "   - Tool: [[create_task]]  \n",
            "   - Parameters:  \n",
            "     `description`: \"Design launch presentation, social media posts, and press releases\"  \n",
            "     `priority`: 1  \n",
            "\n",
            "4. **Create task: Confirm event venue and logistics**:  \n",
            "   - Tool: [[create_task]]  \n",
            "   - Parameters:  \n",
            "     `description`: \"Book venue, arrange AV equipment, and coordinate catering\"  \n",
            "     `priority`: 2  \n",
            "\n",
            "5. **Create task: Train team on launch流程**:  \n",
            "   - Tool: [[create_task]]  \n",
            "   - Parameters:  \n",
            "     `description`: \"Conduct training sessions for sales and support teams on new product features\"  \n",
            "     `priority`: 2\n",
            "\n",
            "=== Plan Execution Phase ===\n",
            "\n",
            "\n",
            "Using general model qwen-plus-0919 to format output:\n",
            "{\n",
            "  \"steps\": [\n",
            "    {\n",
            "      \"tool\": \"schedule_event\",\n",
            "      \"params\": {\n",
            "        \"title\": \"Product Launch Event\",\n",
            "        \"time\": \"2023-11-17T14:00:00\"\n",
            "      }\n",
            "    },\n",
            "    {\n",
            "      \"tool\": \"create_task\",\n",
            "      \"params\": {\n",
            "        \"description\": \"Review and finalize all product features for launch\",\n",
            "        \"priority\": 1\n",
            "      }\n",
            "    },\n",
            "    {\n",
            "      \"tool\": \"create_task\",\n",
            "      \"params\": {\n",
            "        \"description\": \"Design launch presentation, social media posts, and press releases\",\n",
            "        \"priority\": 1\n",
            "      }\n",
            "    },\n",
            "    {\n",
            "      \"tool\": \"create_task\",\n",
            "      \"params\": {\n",
            "        \"description\": \"Book venue, arrange AV equipment, and coordinate catering\",\n",
            "        \"priority\": 2\n",
            "      }\n",
            "    },\n",
            "    {\n",
            "      \"tool\": \"create_task\",\n",
            "      \"params\": {\n",
            "        \"description\": \"Conduct training sessions for sales and support teams on new product features\",\n",
            "        \"priority\": 2\n",
            "      }\n",
            "    }\n",
            "  ]\n",
            "}\n",
            "\n",
            "Call tool execution, execution result:\n",
            "{\n",
            "  \"status\": \"completed\",\n",
            "  \"results\": [\n",
            "    {\n",
            "      \"tool\": \"schedule_event\",\n",
            "      \"result\": \"Event scheduled: Product Launch Event\"\n",
            "    },\n",
            "    {\n",
            "      \"tool\": \"create_task\",\n",
            "      \"result\": \"Task created: Review and finalize all product features for launch\"\n",
            "    },\n",
            "    {\n",
            "      \"tool\": \"create_task\",\n",
            "      \"result\": \"Task created: Design launch presentation, social media posts, and press releases\"\n",
            "    },\n",
            "    {\n",
            "      \"tool\": \"create_task\",\n",
            "      \"result\": \"Task created: Book venue, arrange AV equipment, and coordinate catering\"\n",
            "    },\n",
            "    {\n",
            "      \"tool\": \"create_task\",\n",
            "      \"result\": \"Task created: Conduct training sessions for sales and support teams on new product features\"\n",
            "    }\n",
            "  ]\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "from openai import OpenAI\n",
        "import os\n",
        "import json\n",
        "from typing import Generator\n",
        "\n",
        "class TaskPlanningSystem:\n",
        "    def __init__(self):\n",
        "        # Initialize client connection\n",
        "        self.client = OpenAI(\n",
        "            api_key=os.getenv(\"DASHSCOPE_API_KEY\"),\n",
        "            base_url=os.environ[\"BASE_URL\"]\n",
        "        )\n",
        "        \n",
        "        # System toolset (can be extended as needed)\n",
        "        self.tools = {\n",
        "            \"create_task\": {\n",
        "                \"desc\": \"Create a new task item\",\n",
        "                \"params\": {\"description\": \"str\", \"priority\": \"int\"}\n",
        "            },\n",
        "            \"schedule_event\": {\n",
        "                \"desc\": \"Schedule an event\", \n",
        "                \"params\": {\"title\": \"str\", \"time\": \"datetime\"}\n",
        "            }\n",
        "        }\n",
        "\n",
        "    def generate_plan(self, user_request: str) -> Generator[str, None, None]:\n",
        "        \"\"\"Stream-generated task planning\"\"\"\n",
        "        # Build planning prompt\n",
        "        system_prompt = f\"\"\"You are a senior task planner, please convert the user's requirements into executable steps:\n",
        "        \n",
        "        Available tools (format: [[tool name]]):\n",
        "        {json.dumps(self.tools, indent=2)}\n",
        "        \n",
        "        Output requirements:\n",
        "        1. Use Markdown list format\n",
        "        2. Each step should indicate the tool name\n",
        "        3. Include necessary parameter examples\n",
        "        \"\"\"\n",
        "        model=\"qwq-32b\"\n",
        "        # Initiate streaming request\n",
        "        completion = self.client.chat.completions.create(\n",
        "            model=model,\n",
        "            messages=[\n",
        "                {\"role\": \"system\", \"content\": system_prompt},\n",
        "                {\"role\": \"user\", \"content\": user_request}\n",
        "            ],\n",
        "            stream=True,\n",
        "            temperature=0.3\n",
        "        )\n",
        "\n",
        "        # Print thinking process title\n",
        "        print(f\"\\n\\n=== [Planning Thinking Phase] Using reasoning model: {model}===\")\n",
        "        is_answering = False\n",
        "        # Process streaming response\n",
        "        for chunk in completion:\n",
        "            if chunk.choices:\n",
        "                delta = chunk.choices[0].delta\n",
        "                if hasattr(delta, 'reasoning_content') and delta.reasoning_content is not None:\n",
        "                    # Process thinking process content\n",
        "                    reasoning_content = delta.reasoning_content\n",
        "                    yield reasoning_content\n",
        "                else:\n",
        "                    # Switch to answer output mode\n",
        "                    if delta.content != \"\" and not is_answering:\n",
        "                        print(f\"\\n\\n=== [Planning Generation Phase] Using reasoning model: {model}===\")\n",
        "                        is_answering = True\n",
        "                    # Process answer content\n",
        "                    if delta.content:\n",
        "                        content = delta.content\n",
        "                        yield content\n",
        "\n",
        "    def execute_plan(self, plan: str) -> dict:\n",
        "        \"\"\"Execute the generated task plan\"\"\"\n",
        "        # Call general model to parse the plan\n",
        "        analysis_prompt = f\"\"\"Please parse the following task plan and generate executable instructions:\n",
        "        \n",
        "        Plan content:\n",
        "        {plan}\n",
        "        \n",
        "        Output requirements:\n",
        "        - The returned content must be in JSON format only, do not include other information, do not output code blocks.\n",
        "        - Include tool names and parameters\n",
        "        - Example:\n",
        "            {{\"steps\": [\n",
        "                {{\"tool\": \"create_task\", \"params\": {{\"description\": \"...\"}}}}\n",
        "            ]}}\n",
        "        \"\"\"\n",
        "        model=\"qwen-plus-0919\"\n",
        "        response = self.client.chat.completions.create(\n",
        "            model=model,\n",
        "            messages=[{\"role\": \"user\", \"content\": analysis_prompt}],\n",
        "            temperature=0\n",
        "        )\n",
        "\n",
        "        print(f\"\\n\\nUsing general model {model} to format output:\\n{response.choices[0].message.content}\")\n",
        "        # Parse execution instructions\n",
        "        try:\n",
        "            instructions = json.loads(response.choices[0].message.content)\n",
        "            return self._run_instructions(instructions)\n",
        "        except json.JSONDecodeError:\n",
        "            return {\"error\": \"Instruction parsing failed\"}\n",
        "\n",
        "    def _run_instructions(self, instructions: dict) -> dict:\n",
        "        \"\"\"Actually execute tool calls\"\"\"\n",
        "        results = []\n",
        "        for step in instructions.get(\"steps\", []):\n",
        "            tool = step.get(\"tool\")\n",
        "            params = step.get(\"params\", {})\n",
        "            \n",
        "            # Execute tool call (this is an example implementation)\n",
        "            if tool == \"create_task\":\n",
        "                results.append({\n",
        "                    \"tool\": tool,\n",
        "                    \"result\": f\"Task created: {params.get('description')}\"\n",
        "                })\n",
        "            elif tool == \"schedule_event\":\n",
        "                results.append({\n",
        "                    \"tool\": tool,\n",
        "                    \"result\": f\"Event scheduled: {params.get('title')}\"\n",
        "                })\n",
        "            else:\n",
        "                results.append({\"error\": f\"Unknown tool: {tool}\"})\n",
        "        \n",
        "        return {\"status\": \"completed\", \"results\": results}\n",
        "\n",
        "# Usage example\n",
        "if __name__ == \"__main__\":\n",
        "    PlanningSystem = TaskPlanningSystem()\n",
        "    \n",
        "    # Example user request\n",
        "    user_request = \"Please schedule next week's product launch and create related preparation tasks\"\n",
        "    \n",
        "    # Planning generation phase\n",
        "    plan_stream = PlanningSystem.generate_plan(user_request)\n",
        "    generated_plan = []\n",
        "    for chunk in plan_stream:\n",
        "        print(chunk, end=\"\", flush=True)\n",
        "        generated_plan.append(chunk)\n",
        "    \n",
        "    full_plan = \"\".join(generated_plan)\n",
        "    \n",
        "    print(\"\\n\\n=== Plan Execution Phase ===\")\n",
        "    result = PlanningSystem.execute_plan(full_plan)\n",
        "    print(f\"\\nCall tool execution, execution result:\")\n",
        "    print(json.dumps(result, indent=2, ensure_ascii=False))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3cf7690c",
      "metadata": {},
      "source": [
        "The above code implements the following core interaction process. The code mainly uses the qwq-32b model for streaming planning generation, then uses qwen-plus-0919 for instruction parsing, and finally decouples tool invocation through the _run_instructions method, making it easier to extend in the future:\n",
        "\n",
        "<a href=\"https://img.alicdn.com/imgextra/i4/O1CN01OOAmtb1flZx0MxbUn_!!6000000004047-2-tps-1623-603.png\" target=\"_blank\">\n",
        "<img src=\"https://img.alicdn.com/imgextra/i4/O1CN01OOAmtb1flZx0MxbUn_!!6000000004047-2-tps-1623-603.png\" width=\"1000\">\n",
        "</a>\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "30451423",
      "metadata": {},
      "source": [
        "## ✅ Summary\n",
        "\n",
        "Through  this section, you have  gained a solid understanding of prompts that are suitable for both general LLMs and reasoning LLMs. The flexible application of these techniques ensures a baseline level of quality in AI-generated responses. However, as AI technology continues to evolve, these methods may also change—such as when reasoning models no longer require chain-of-thought prompting to achieve optimal performance.What’s most important to understand is that the core purpose of prompt engineering remains the same: to clearly express your core needs and provide rich, relevant background information (context). This is essential for effectively leveraging LLMs and ensuring they deliver accurate, meaningful results.\n",
        "\n",
        "In real-world LLM applications, domain experts often play a key role in designing prompts. Therefore, hardcoding prompts directly into your code should be avoided. Instead, consider making prompts configurable or even building an entirely configurable application workflow. This allows domain experts—especially those without technical backgrounds—to actively participate in refining prompts and shaping the overall process.\n",
        "\n",
        "Alibaba Cloud's Model Studio offers a powerful application building capability, enabling users to write prompts and visually design complex application flows directly on the platform. This makes it an ideal tool for LLM projects that involve collaboration with non-technical domain experts.\n",
        "\n",
        "In the next section, you will learn about automated evaluation methods to assess the performance of Q&A bots. You’ll explore how to use quantitative metrics to measure the effectiveness of your prompts and continuously improve your AI-driven applications."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3166b70c",
      "metadata": {},
      "source": [
        "## 🔥 Quiz\n",
        "\n",
        "### 🔍 Multiple choice question\n",
        "\n",
        "<details>\n",
        "<summary style=\"cursor: pointer; padding: 12px; border: 1px solid #dee2e6; border-radius: 6px;\">\n",
        "<b>Which of the following prompt elements is used to explicitly define the task that the LLM needs to complete❓ (Select 1.)</b>\n",
        "\n",
        "- A. Role\n",
        "- B. Audience\n",
        "- C. Objective\n",
        "- D. Context\n",
        "\n",
        "**[Click to view answer]**\n",
        "</summary>\n",
        "\n",
        "<div style=\"margin-top: 10px; padding: 15px; border: 1px solid #dee2e6; border-radius: 0 0 6px 6px;\">\n",
        "\n",
        "✅ **Reference Answer: C**\n",
        "📝 **Explanation**:\n",
        "- The Objective explicitly defines the operation or outcome that the large language model (LLMs) is required to perform. Other options do not directly define the task itself.\n",
        "- Role defines the identity that the LLMs should assume, Audience specifies the target group, and Context provides background information.\n",
        "\n",
        "</div>\n",
        "</details>\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "<details>\n",
        "<summary style=\"cursor: pointer; padding: 12px; border: 1px solid #dee2e6; border-radius: 6px;\">\n",
        "<b>Suppose you want to use a large language model to generate a description of the solar system suitable for third-grade students. Which of the following user_query designs is more reasonable❓ (Select all that apply.)</b>\n",
        "\n",
        "\n",
        "- A. user_query=\"Write about the solar system.\"\n",
        "- B. user_query=\"Tell me about the solar system.\"\n",
        "- C. user_query=\"Explain the solar system as if you were talking to a third-grade student.\"\n",
        "- D. user_query=\"Write a short and engaging description of the solar system for a third-grade audience, focusing on key planets and their characteristics.\"\n",
        "- E. user_query=\"What is the solar system?\"\n",
        "\n",
        "**[Click to view answer]**\n",
        "</summary>\n",
        "\n",
        "<div style=\"margin-top: 10px; padding: 15px; border: 1px solid #dee2e6; border-radius: 0 0 6px 6px;\">\n",
        "\n",
        "✅ **Reference Answer: CD**\n",
        "📝 **Explanation**:\n",
        "- Options C and D clearly specify the target audience and provide requirements for content and style.\n",
        "\n",
        "</div>\n",
        "</details>\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "<details>\n",
        "<summary style=\"cursor: pointer; padding: 12px; border: 1px solid #dee2e6; border-radius: 6px;\">\n",
        "<b>[Multi-Choice]Suppose you want to use a large language model to generate a description of the solar system suitable for third-grade students. Which of the following user_query designs are most reasonable❓ (Select all that apply.)</b>\n",
        "\n",
        "- A. Provide simple and clear task instructions\n",
        "- B. Supplement with sufficient background information\n",
        "- C. If there are specific user groups or task requirements, set roles or audiences\n",
        "- D. Use chain-of-thought prompts to make the reasoning process more rigorous and reliable\n",
        "- E. When descriptions are too abstract or cannot be accurately described, clarify them by adding examples\n",
        "\n",
        "**[Click to view answer]**\n",
        "</summary>\n",
        "\n",
        "<div style=\"margin-top: 10px; padding: 15px; border: 1px solid #dee2e6; border-radius: 0 0 6px 6px;\">\n",
        "\n",
        "✅ **Reference Answer: ABCE**\n",
        "📝 **Explanation**:\n",
        "- When using reasoning models, it is advisable to avoid using chain-of-thought prompting, as this may reduce the effectiveness of reasoning.\n",
        "\n",
        "</div>\n",
        "</details>\n"
      ]
    }
  ],
  "metadata": {
    "jupytext": {
      "cell_metadata_filter": "-all",
      "main_language": "python",
      "notebook_metadata_filter": "-all"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.12.10"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}
